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METHOD AND SYSTEM FOR ERROR DIFFUSION WITH A 
PLURALITY OF ERROR MEASURES 

BACKGROUND OF THE INVENTION 

Field of the Invention 

The present invention generally relates to halftoning techniques in 
printers and displays and, more particularly, to a method and apparatus for 
halftoning with a display or printing medium such that the displayed (or 
printed) image pixels have different characteristics depending on conditions of 
the viewing environment such as viewing angle, temperature and lighting 
conditions. 

The inventive halftoning method and apparatus generate a halftoned 
output such that a different image is displayed under each condition of the 
viewing environment. 

Description of the Related Art 

Most printers and displays show images which look relatively the same 
under different viewing conditions. However, this is not true for all printing 
technologies and displays. For instance, for a type of LCD (Liquid Crystal 
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Display) display commonly used in laptop computers, the image pixels change 
color and brightness drastically as the viewing angle is changed. Other 
examples include inks which change color or density, as the temperature is 
changed, or inks which change color or density when the printed paper is 
5 illuminated with ultraviolet light. 

Given images Ni, N 2? a problem arises of how to generate an image 
such that it looks like Ni under viewing condition 1 , it looks like N 2 under 
viewing condition 2, etc. 

Further, because images printed by a printer or displayed on a display 

1 0 can change color characteristics as the conditions of the viewing environment 
change, another problem arises as to how to generate images which look the 
same even as these viewing conditions change. 

Other systems exist which display two or more images depending on 
viewing angles, such as those used on decals, baseball cards, etc. There are 

15 several differences between these systems and the present invention. First, 
these printed papers in the conventional systems are designed to display 
different images at different angles, and thus no special processing of the 
image is needed. In the present invention, the display (or printing medium) is 
not specifically designed to display different images at different viewing 

20 conditions, and therefore the present invention teaches methods to process the 
images so that the display will show different images under different viewing 
conditions. 
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Second, the conventional systems mainly deal with different viewing 
angles, whereas the present invention can also be applied to showing different 
images under different lighting conditions, temperature, etc. 

Third, the conventional systems are mainly applicable to printed 
images, whereas the present invention is also applicable to displays which can 
accommodate video. 

SUMMARY OF THE INVENTION 

In view of the foregoing and other problems of the conventional 
methods and systems, an object of the present invention is to provide a system 
and method which can generate an image such that it looks like Ni under 
viewing condition 1, it looks like N 2 under viewing condition 2, etc. 

Another object is to maintain color characteristics of images printed by 
a printer or displayed on a display as the conditions of the viewing 
environment change, and to generate images which look the same even as 
these viewing conditions change. 

In a first aspect of the present invention, a method (and system) for 
producing a halftoned image, includes calculating errors corresponding to a 
plurality of different viewing conditions of a halftone image, and minimizing a 
function of the errors, such that the halftoned image appears as a different 



YOR920000354US1 



3 



image under different viewing conditions. 

In a different aspect, the halftoned image appears as the same image 
under different viewing conditions. 

Conditions of the viewing environment include variables such as 
5 temperature, viewing angle, humidity, lighting conditions, etc. Herein, these 
will be termed viewing conditions for short. A specific viewing condition 
could be viewing the display (or printed output) at a specific angle (or angle 
ranges) or the like. 

In the following discussion, whenever a display" is referred to, it 
1 0 should be understood that this includes any output device, such as a laptop 
LCD display, printer, cathode ray tube (CRT), etc. 

At each pixel, the display can display one color out of a set of colors 
{ci, c 2 , c„}. 

Given a color Ci, the actual color seen by the viewer under viewing 
1 5 condition j will be denoted ay. a,j can be a scalar for grayscale or a 

three-dimensional vector for color in RGB, LAB, CMY, XYZ or other color 
spaces. An introduction to color spaces can be found in "Color Technology 
for Electronic Imaging Devices", H. R. Kang, SPIE Press, 1997. It is clear 
how this formulation is also valid for other high dimensional color spaces. 
20 In classical error diffusion halftoning, halftoning errors from previous 

pixels are added to the input pixel resulting in the modified input. The 
displayable color closest to the modified input is then displayed. The 
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difference between the modified input and the printed (displayed) color is the 
halftoning error which is propagated to neighboring pixels. 

Error diffusion and other halftoning techniques are described in the 
book "Digital Halftoning" by R. Ulichney, MIT Press, 1987. 
5 Assume that images Ni, N 2 , ... are given. By using instead of d and 

choosing N,, an error ej corresponding to the viewing condition] for halftoning 
image Nj can be obtained. In the present invention, a halftoned image is 
generated which attempts to minimize the aggregate error which is a function 
of the e/s. 

1 0 Thus, with the unique and unobvious aspects of the present invention, 

an image can be generated such that it looks like Ni under viewing condition 

1, it looks like N 2 under viewing condition 2, etc. 

Further, in a different aspect, color characteristics can be maintained of 

images printed by a printer or displayed on a display as the conditions of the 
15 viewing environment change. Thus, images can be generated which look the 

same even as the viewing conditions change. 

Hence, with the invention, several different error measures can be 

considered and the color selected minimizes the error measures jointly by 

minimizing an aggregate of the error measures. 
20 Typically, error diffusion methods result in selecting one color from a 

limited set of colors at each pixel. Traditionally, error diffusion tries to 

minimize the error between what image is output and what the original image 
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actually is. Typically, an attempt is made to pick the right color such that at a 
distance the output appears the same as the original 

However, in contrast to the conventional methods, there are a plurality 
(e.g., two or more) of errors considered in the present invention. For example, 

5 there may be one error looking at an image at one viewing condition (e.g., 

angle, temperature, lighting condition, etc.) and displaying it, and there may be 
another error in viewing the same image at another viewing condition (e.g., 
angle, temperature, etc.) and displaying it. Thus, there are two errors, but the 
conventional methods attempt to minimize only one of them. 

10 In the inventive method, the error diffusion is performed by 

minimizing an aggregate of the errors (e.g., at the same time), not just one 
error or the other as in conventional error diffusion. Thus, colors are selected 
to minimize the errors at each pixel. 



BRIEF DESCRIPTION OF THE DRAWINGS 

1 5 The foregoing and other objects, aspects and advantages will be better 

understood from the following detailed description of a preferred embodiment 
of the invention with reference to the drawings, in which: 

Figure 1 A illustrates a flow diagram of a method for performing error 
diffusion with a plurality of error measures according to a preferred 
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embodiment of the invention ; 

Figure IB illustrates the preferred embodiment of the invention of 
Figure 1 in pseudo-code form; 

Figure 2 A is an image generated from the prescribed algorithm viewed 
on a LCD display from one angle; 

Figure 2B is the same image as Figure 2A viewed from another angle; 

Figure 2C is a color image generated from the prescribed algorithm 
viewed on an LCD display from one angle; 

Figure 2D is the same image as Figure 2C viewed from another angle 
(in contrast to Figures 2A-2B, the same image is embedded in the two viewing 
angles so that the same image of a young woman appears under both viewing 
angles.); 

Figure 3 shows curves for an and aj 2 which would work well to embed 
two images; 

Figure 4 illustrates an exemplary information handling/computer 
system for use with the present invention; and 

Figure 5 illustrates a storage medium 500 for storing steps of the 
program for the method according to the present invention. 
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DETAILED DESCRIPTION OF A PREFERRED 
EMBODIMENT OF THE INVENTION 



Referring now to the drawings, and more particularly to Figures 1 A-5, 
there are shown preferred embodiments of the method and structures 
5 according to the present invention. 

Figure 1 A illustrates a flow chart of the inventive method 1 00, and 
Figure IB illustrates the inventive method in pseudo-code form. 

In Figure 1 A- IB, there are three loops in the process such that a loop is 
made through each column k and rows 1 of the image (e.g., through each 
10 pixel). 

That is, in step 1 10, for each pixel of the image (e.g., at the intersection 
of the columns and rows), a loop is made through each viewing condition. In 
the example of Figures 2A-2B, there are only two viewing conditions shown 
for simplicity (e.g., one showing the baboon and one showing the woman). 

15 In step 120, for each viewing condition, a modified input Mj is created. 

In error diffusion, instead of using an input, a modified input is used (created) 
by adding to the current input pixel N a weighted sum (w,) of errors (ej) that 
have been generated in the past (e.g., errors propagated through). 

Once the modified inputs are generated, then, in step 130 for example, 

20 the aggregate error is calculated for each possible output color. The aggregate 
error can be chosen as the sum of the errors for each viewing condition. 
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In step 140, the color for the halftone image output pixel is chosen as 
the output color which has a small (or the lowest) aggregate error. 

In step 150, the error e s that can occur by selecting a particular output 
then is found for each viewing condition j. That is, a particular output is 
5 subtracted from the modified output M. 

Then, in step 160, for each j, an upper and lower bound is applied to 
the error such that the error does not grow too large in magnitude. Thus, the 
error is clamped at an upper bound and a lower bound if it becomes too large. 
Hence, if the error is larger than the upper bound, it is clamped at the upper 
1 0 bound value. A similar operation is performed at the lower bound if the error 
is smaller than the lower bound. 

Finally, in step 170, an output of the halftone image is generated such 
that the appearance of the image changes as the viewing conditions change. 
Hence, given images Ni, N 2 , the system and method of the present 
1 5 invention create a halftoned image which looks like image Nj under viewing 
condition j for each j. The images Ni, N 2 , will be termed as being 
embedded into the halftoned image. N,(k,l) will be denoted as the (k,l)th pixel 
of image Nj. Each pixel of Nj is a scalar or a vector, depending on whether the 
image is a grayscale image or a color image. 
20 Furthermore, the pixels of each image Nj can be represented in 

different color spaces (i.e., Ni can be in RGB space, N 2 can be in LAB space, 
N 3 can be used in CMY space, etc.). For each image, the corresponding colors 
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a y and Mj(k ? l) are represented in the same color space as the image. Using the 
notation as above, the modified input for viewing condition j is defined as 

H(k ? i) = NjOU) + Jw^v^Gc^l-v) 

where ej(k, 1) is the error corresponding to viewing condition j at the 
5 (k, l)th pixel. The aggregate error in the preferred embodiment at the (k, l)th 
pixel for choosing d as the output pixel color is given by: 

Eti(iJcl) = 2|M J <k4)-aol 

This definition of Err is based on a Li norm. It is clear how other 
definitions of Err can be possible. For example, Err based on a L 2 norm is 
10 defined as: 

Eir(iXl) = (E|^(k,l>a ij | 2 ) % 

and Err based on a weighted Li norm is defined as 
Err(i ? k ? l) = Sn j |M j (k ? l)-a u | 

where n, are the weighting terms. 
1 5 The output pixel color at the (k,l)th pixel is then defined as: 

o(k,l) = Ct where t = argmin Err(i,k 3 l) 
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(i.e., o(k, 1) is the printable color c t which minimizes the aggregate 
error Err). 

The error e,(k,l) is defined as Mj(k,l) - a mj if and only if o(k,l) = c m . 
Wj is the error weights matrix corresponding to the error diffusion 
5 algorithm. For instance, if Jarvis error diffusion (e.g., see Ulichney mentioned 
above) is used for each j 5 then Wj will be defined as shown below in Table 1, 

TABLE 1 



u 



w,(u,v) 


2 


1 


0 


1 


2 


0 


0 


0 


0 


7/48 


5/48 


1 


3/48 


5/48 


7/48 


5/48 


3/48 


2 


1/48 


3/48 


5/48 


3/48 


1/48 



Thus, Table 1 shows the weighting factors that can be exemplary used, 
10 To prevent the error from growing too large, if e,(k,l) > UPBOUNDj then 
ej(k,I) is set to UPBOUNDj. Similarly, if ej(k,l) < LOBOUNDj then e,{k,l) is 
set to LOBOUNDj. UPBOUNDj 10 and LOBOUNDj 10 are set for all j in a 
preferred embodiment. 

It is clear to one skilled in the art that the present invention can be 
1 5 adapted to many variations of error diffusion, such as using different error 
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weights, or using error weights which are input dependent (e.g., see U.S. 
Patent No. 6,006,01 1, incorporated herein by reference) etc. 

In general, for printers, the number of printable colors is small (i.e., 2 
for a black-and-white printer, and 8 for a CMYK bilevel color printer). 

In general, for displays, the list of displayable colors can be quite large. 
For 8-bit grayscale, the number of displayable colors can be as much as 256 
which is still manageable. For 24-bit RGB color, the number of displayable 
colors can be too large to implement the present invention efficiently. In that 
case, the images can be split into their RGB channels (or LAB channels or any 
appropriate color channels) and apply the present invention to each channel 
independently. In this case, all images should be expressed in the same color 
space. Another solution is to restrict the number of displayable colors to a 
manageable size, which depends on the processing speed of the hardware 
responsible for implementing the present invention. 

The present invention is shown in Fig. IB in pseudo-code form. As 
mentioned above, the halftoned image is given by o, i.e., o(k,l) is the (k,l)th 
pixel of the output halftoned image. 

One of the applications of the present invention is to display different 
images depending on the viewing conditions. 

Another application is to display images of warning messages for 
certain viewing conditions, such as when the temperature is too high, the 
viewing angle is too low, etc. 
YOR920000354US1 12 



The present invention also can be used to limit the viewing conditions 
under which the intended image can be viewed by embedding other images for 
the unintended viewing conditions. 

Another application is in counterfeit protection, where 
temperature-sensitive inks are used and a second image only appears when the 
printed paper is heated up by placing a finger on it. Such technology is 
currently used, but the second image is usually very simple and bilevel, such 
as an image of the word "authentic". The present invention allows more 
complicated multilevel images to be embedded as a security feature. Similarly 
the same can be done with inks which are sensitive to ultraviolet light and the 
second image will appear when the document is illuminated with a ultraviolet 
light source. 

Fig. 2A shows a halftoned image generated from the present invention 
when viewed from one viewing angle on an LCD display for the case when 
the image is a grayscale image. That is, Fig. 2A shows an image of a baboon 
Two viewing conditions are used corresponding to two different viewing 
angles and 4 grayscale output colors are used as shown in Table 2 below. 
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TABLE 2 



i 





1 


2 


3 


4 


Ci 


0 


40 


180 


255 


a;i 


0 


40 


180 


255 


a l2 


0 


115 


3 


255 



Thus, Table 2 shows the 4 output colors d and how they appear under 
viewing condition 1 (aa) and viewing condition 2 (a^). The two viewing 
5 conditions are the viewing angles as defined by the angle between the viewer's 
viewing direction and the normal direction (the perpendicular direction) of the 
image. 

Fig. 2B shows the same halftoned image viewed from another viewing 
angle. As shown, the image is now of a young woman. Another application is 
10 to increase the robustness of the image under varying viewing conditions (i.e., 
the printed (or displayed) image should look the same under different viewing 
conditions). In this case, the images Ni, N 2 , ... are chosen to be the same 
image. 

For example, a halftoned image was generated where the same color 
15 image was embedded in the two viewing angles. In this case, the image is 
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into its RGB channels and the algorithm is applied to each color channel using 
Table 2. Fig. 2C and Fig. 2D show this image as viewed under the two 
viewing angles respectively. One sees that the same image of a young woman 
appears under the two viewing conditions. 

5 Since video can be decomposed as a series of still images, the present 

invention also can be used to embed several video sequences in a display, each 
of which can be seen under different viewing conditions. 

If the color does not change much under a different viewing condition, 
then it is very unlikely that the image will appear substantially different under 

1 0 different viewing conditions. Therefore, to successfully embed several very 
different images, the sequences {aj as a function of i should be very different 
for different j. For instance, in the example above, {a,i} is monotonic, while 
{a i2 } is not. In the grayscale case, preferably, the following condition should 
hold: for each viewing condition j, assign one of the two labels "light" or 

1 5 "dark". For any such combination of labels for the viewing conditions, there 
exists a index i such that a y for each j corresponds to the chosen label, i.e. a^ is 
dark if viewing condition j has label "dark" , a y is light if viewing condition] 
has label "light" , for each viewing condition j. For example, for two viewing 
conditions, the two curves shown in Fig. 3 will work well. 

20 Figure 3 shows a scheme (conditions) in which the images will appear 

different and will allow the image(s) to look different under two different 
viewing conditions. In particular, Figure 3 illustrates two curves showing how 
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the gray levels (color) of the image pixels depend upon the input i for the two 
viewing conditions. 

That is, a relationship is shown between the input value and the 
perceived color of the image pixels for the two viewing conditions. In an 
5 idealized image pixel, a linear relationship would exist in which as the input 
becomes greater, the color becomes darker. Hence, for example, an is the gray 
level of a normalized pixel under one viewing condition. Thus, as the 
brightness is increased (e.g., the input), the darkness of the pixel is increased 
monotonically. 

10 The second curve illustrates the same image pixel under a second, 

different viewing condition. As shown, a small amount of input (e.g., turning 
the image pixel on slightly) will result in very dark pixel, and when the input 
is increased more then the pixel becomes lighter until the very end of the 
curve. Hence, in the second curve, the input value and the perceived color 

1 5 have almost an inverse relationship. As shown, at some point of the 

horizontal axis i, there is a case of when the first viewing condition is low and 
the second condition is high, or vice versa, or some cases when the first and 
second viewing conditions are both low (e.g., at the beginning of the curves 
where i is almost 0) and are both high (e.g. at the end of the curve where i is 

20 large). 

As shown below in Table 3, for the following values of Cj and a*,: 
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TABLE 3 

i 





1 


2 


3 


4 


Ci 


0 


1 


254 


255 




0 


0 


255 


255 


an 


0 


255 


0 


255 



there is a simple interpretation of the resulting invention. That is, two images 
are independently halftoned by bilevel error diffusion. At each pixel position, 
5 the pixels of the two halftoned images form a pixel pair of one of the 

following four forms: (0, 0) ? (0, 255), (255, 0) and (255, 255), from which the 
output color is chosen as 0, 1, 254, 255, respectively. 

While the overall methodology of the invention is described above, the 
invention can be embodied in any number of different types of systems and 
10 executed in any number of different ways, as would be known by one 
ordinarily skilled in the art. 

For example, as illustrated in Figure 4 , a typical hardware 
configuration of an information handling/computer system for use with the 
invention. In accordance with the invention, preferably the system has at least 
1 5 one processor or central processing unit (CPU) 411 and more preferably 
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several CPUs 411. The CPUs 41 1 are interconnected via a system bus 412 to 
a random access memory (RAM) 414, read only memory (ROM) 416, 
input/output (I/O) adapter 418 (for connecting peripheral devices such as disk 
units 421 and tape drives 440 to the bus 412), user interface adapter 422 (for 
5 connecting a keyboard 424, an input device such as a mouse, trackball, 

joystick, touch screen, etc. 426, speaker 428, microphone 432, and/or other 
user interface device to the bus 412), communication adapter 434 (for 
connecting the information handling system to a data processing network such 
as an intranet, the Internet (World- Wide- Web) etc.), and display adapter 436 

10 (for connecting the bus 412 to a display device 438). The display device could 
be a cathode ray tube (CRT), liquid crystal display (LCD), etc., as well as a 
hard-copy printer (e.g., such as a digital printer). 

In addition to the hardware/software environment described above, a 
different aspect of the invention includes a computer-implemented method for 

1 5 producing a halftone image. This method may be implemented in the 
particular environment discussed above. 

Such a method may be implemented, for example, by operating a 
computer, as embodied by a digital data processing apparatus, to execute a 
sequence of machine-readable instructions. These instructions may reside in 

20 various types of signal-bearing media. 

Such a method may be implemented, for example, by operating the 
CPU 411 (Figure 4), to execute a sequence of machine-readable instructions. 

YOR920000354US1 18 



These instructions may reside in various types of signal-bearing media. 

Thus, this aspect of the present invention is directed to a programmed 
product, comprising signal-bearing media tangibly embodying a program of 
machine-readable instructions executable by a digital data processor 
5 incorporating the CPU 41 1 and hardware above, to perform the above method. 
This signal-bearing media may include, for example, a RAM (not 
shown in Figure 4) contained within the CPU 41 1 or auxiliary thereto as in 
RAM 414, as represented by a fast-access storage for example. Alternatively, 
the instructions may be contained in another signal-bearing media, such as a 
10 magnetic data storage diskette 500 (e.g., as shown in Figure 5), directly or 
indirectly accessible by the CPU 411. 

Whether contained in the diskette 500, the computer/CPU 41 1, or 
elsewhere, the instructions may be stored on a variety of machine-readable 
data storage media, such as DASD storage (e.g., a conventional "hard drive" 
15 or a RAID array), magnetic tape, electronic read-only memory (e.g., ROM, 
EPROM, or EEPROM), an optical storage device (e.g., CD-ROM, WORM, 
DVD, digital optical tape, etc.), paper "punch" cards, or other suitable 
signal-bearing media including transmission media such as digital and analog 
and communication links and wireless. In an illustrative embodiment of the 
20 invention, the machine-readable instructions may comprise software object 
code, compiled from a language such as "C", etc. 

Thus, with the unique and unobvious aspects of the present invention, 
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a method (and system) are provided which can generate an image such that it 
looks like Ni under viewing condition 1, it looks like N 2 under viewing 
condition 2, etc. 

Further, color characteristics of images printed by a printer or 
5 displayed on a display can be maintained as the conditions of the viewing 
environment change, and which can generate images which look the same 
even as these viewing conditions change. 

Further, in the invention, a halftoned image is generated which 
attempts to minimize the aggregate error which is a function of the ej_s. 
1 0 Thus, with the unique and unobvious aspects of the present invention, 

an image can be generated such that it looks like Ni under viewing condition 
1, it looks like N 2 under viewing condition 2, etc. 

Further, color characteristics can be maintained of images printed by a 
printer or displayed on a display as the conditions of the viewing environment 
1 5 change. Thus, images can be generated which look the same even as the 
viewing conditions change (e.g., an image of a baboon under one viewing 
condition would look the same under another viewing condition). 

Hence, with the invention, several different error measures can be 
considered and, for example, several different colors can be considered such 
20 that the color selected has to be constrained. Further, the error diffusion 
method of the invention tries to minimize the error between what image is 
output and what the original image actually is. In contrast to the conventional 
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methods, there are a plurality (e.g., two or more) of errors considered. 
Moreover, in the inventive method, the error diffusion is performed by 
minimizing an aggregate of the errors (e.g., at the same time), not just one 
error or the other as in conventional error diffusion. Thus, colors are selected 
to minimize the errors at each pixel 

While the invention has been described in terms of a single preferred 
embodiment, those skilled in the art will recognize that the invention can be 
practiced with modification within the spirit and scope of the appended 
claims. 
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